In the traditional von Neumann model of computing, instructions are executed one at a time in sequence. A central feature of this model is the instruction pointer. It points to the instruction that is currently executing. Each instruction has a unique successor which is usually at the following address that immediately follows the current instruction pointer address. To execute the next instruction in a sequence, the instruction pointer is usually incremented. Execution of that next instruction is then imperative. The sequence of instructions executed in a program is often referred to as a thread of computation.
Parallel processing machines are not limited to performing one thread at a time. They perform a number of threads simultaneously. They achieve this because they comprise multiple sequential von Neumann machines, each of which performs a single thread of computation.